RESEARCH PAPERS 


A QUALITATIVE PERSPECTIVE: PRE-SERVICE TEACHERS DISCUSS 
EDUCATIONAL TECHNOLOGY ONLINE 


By 

ALEXANDRU SPATARIU * DENISE L. WINSOR ** 

* Assistant Professor, Georgetown College, Kentucky. 

** Assistant Professor, University of Memphis, Tennessee. 

ABSTRACT 

Although there is a push in education to increase the amount of technologies used in the classroom much of the recent 
literature emphasizes a need to investigate ways to improve the instructional methods used when incorporating 
technology in education. The focus of the present research is to investigate an instructional technique used in an 
asynchronous online discussion (i.e., prompts versus no prompts). This research investigates pre-service teachers' 
positions on the use of technology in K-12 classrooms. Participants were students enrolled in an online Educational 
Psychology course at a Southwestern urban university; the online discussion transcripts were the data source for this 
qualitative research design. The treatment group received specific directions and feedback and the control group 
received very limited directions and feedback. The results of the study indicated that given specific guidelines and 
expectations for the discussion, the treatment group outperformed the control group in every category consistently 
within the small group analysis and the between group analysis. In addition, the in-depth analysis points to several 
findings with regard to how pre-service teachers view Educational Technology in schools, based on the dilemma posed 
in the online discussion forum. 

Keywords: Online Asynchronous Discussions, Argumentation Discourse, Educational Technology. 


INTRODUCTION 

Online discussion forums are constantly employed in 
education courses to enhance student discourse leading 
to an enriched understanding of course content (Chinn & 
Waggoner, 1992; Spatariu, Hartley, Schraw, Bendixen, & 
Quinn, 2007), Factors such as background knowledge, 
instructional tasks, group structure, personal characteristics, 
assigned reading, and scaffolding are key factors to 
effective online discourse (Erkens, 1997; Stein, & Miller, 
1990; Veerman, Andriessen, & Kanselaar, 1999; Veerman & 
Treasure-Jones, 1999). Researchers contend that more 
qualitative analysis of discussions is necessary to 
understand the intricacies of discourse construction and 
results within education courses (Marra, Moore, Klimczak, 
2004; Gibson, 2003; Rourke & Szabo, 2002; Winiecki, 2003). 

Spatariu et al. (2006) explored the influence of a discussion 
leader procedure with specific instructions on the quality of 
online argumentation and interactivity from a quantitative 
perspective. The results of the study indicated that students 


participating in groups receiving specific instructions 
produced better argumentation discourse and exhibited 
increased interactivity patterns. However, the particular 
participant reactions both at individual and group level 
were not captured. The researchers examined in-depth the 
reactions and ideas of each participant as they 
contributed to the construction of arguments based on the 
issue at hand, whether or not schools should adopt new 
educational technologies, which led to instrumental 
information that can be used for future design and 
structure of online discussion groups. 

Argumentation for the purpose of this study, as well as 
defined by other researchers, refers to making convincing 
claims backed up by sound evidence and understanding 
various aspects of an issue involving recognizing conflict 
presence and being willing to engage in debate (Stein & 
Miller, 1991; Coirier, Andriessen, & Lucille 1999; Golder & 
Pouit, 1999). The aim of the study is to investigate the nature 
of the discussion among participants in the same group 
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and collectively between groups. The authors wanted to 
conduct an in-depth analysis of the participant's reactions, 
ideas, and positions to identify how they are influenced by 
the group dynamics and the leader instructions. Thus, the 
study answered two main research questions: 

• Does online instruction impact student discourse during 
asynchronous online discussions? 

• What are the similarities/differences within the treatment 
and control groups? If there are differences between the 
treatment and control groups, to what extent are the results 
between groups different? 

This paper is divided into four sections. First, they present a 
succinct overview of existing qualitative discourse analysis 
studies. Second, they provide a descriptive overview of the 
methods used and describe the methodology and data 
analysis procedures. Third, they report the results and 
analyze the findings. Finally, based on the results they 
discuss instructional interventions that can be instrumental 
in boosting the quality of online arguments, 

Qualitative analysis and online discussions 

Although there is a growing body of quantitative research 
on arguments in online settings not much has been done in 
the realm of qualitative research in this area (Tesch,l 990). 
Existing quantitative research studies show that 
argumentation is related to factors such as group 
dynamics (Chang, 2009; Clark, D'Angelo, & Menekse, 
2009; Jeong, 2004), message labeling (ChanLin, Chen, & 
Chan, 2009; Jeong & Joung, 2003; Valcke, De Weaver, 
Zhu, & Deed, 2010), note starters (Nussbaum, Hartley, 
Sinatra, Reynolds, & Bendixen, 2002), discussion instructions 
(Heejung, Sunghee, & Keol, 2009; Jorczak,, & Bart, 2009; 
Nussbaum, 2005; Richardson, & Ice, 2010; Spatariu, etal., 
2007), and personal characteristics of learners (Bendixen, 
Hartley, Sas, & Spatariu, 2003; Nussbaum et al., 2002; So, 
2009; Zhang, Kohler, & Spatariu, 2009). 

A number of recent studies employ discourse analysis 
related to the use of online discussions. Marijke, Hilde, Bram, 
& Martin (2010) used grounded theory to explore cognitive 
processes of undergraduate students engaged in online 
discourse in science education classes. The results 
identified a number of issues such as strategy use, 
intervention scope, interaction, previous discussions 


experience all of them mostly related to peer tutoring. 
Based on these some suggestions were made to help the 
tutors feel more at ease as they could experience 
uneasiness related to lack of subject matter expertise or 
timing of intervention. The purpose of the current study was 
not to focus just on the discussion leader participation; it 
could be that such factors play a role especially in 
argumentative situations involving peers as leaders. 
Another study Baran & Correira (2009) applied qualitative 
inquiry methods to uncover strategies used by graduate 
educational technology students to manage challenges 
resulting from instructor dominated online discussions. 
Results of the study showed, strategies that were 
inspirational of practice oriented led to a more relaxed and 
conducive learning participatory environment. In the 
present study the treatment group was given specific 
instructor guideline, therefore expectations were imposed 
on the participants which presented possible pressure. 
However, the instructions were posted once then the 
groups took the discussions further with instructor 
intervention or probing. The control group was provided 
with less instructor guidance and feedback. A qualitative 
study on online discourse in a math and science pre¬ 
service course (Liang, Ebenzer, & Yost, 2010) revealed that 
collaboration was employed in developing research 
proposals. Although the online discourse enhanced the 
class overall, the critical aspect of evaluating others view 
points through scientific inquiry was not present. This study 
prompted participating students to openly think critically 
and debate an important educational aspect of their 
career as future teachers. 

Methodology 

Technology is a hot topic in education and receives 
positive and negative attention in the literature from a 
variety of perspectives (i.e., students, teachers, parents, 
government). In this study the researchers investigate 
asynchronous online discussion from an instructional 
perspective. In this section they present their method for this 
research and provide an overview of the method of 
analysis. They conducted preliminary quantitative analysis 
which revealed statistical significance; these results are in¬ 
line with recent research literature; and guided more in- 
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depth qualitative analysis. They will discuss the quantitative 
results that coincide with the qualitative data analysis only 
to identify the foundation for this fine-grained analysis; and 
demonstrate their motivation to examine the data more 
closely, using content analysis, which is the focus of this 
research, 

Participants 

The participants were 44 pre-service teachers enrolled in 
undergraduate educational technology courses at a large 
Southwestern university in the United States. Demographic 
and discussion data was collected via WebCT, an online 
management system tool. All participants were randomly 
assigned to either a treatment position (N=23), 6 small 
discussion groups with a leader and pre-determined 
argumentation instructions, or control position (N=23), 6 
small discussion groups with a leader but no instructions. 
Study procedures involved completion of consent forms 
and demographic profile. Neither student leaders nor 
group members had any knowledge of the assessed 
discussion outcomes as per this study. 

Procedure 

This study investigates pre-service teachers by proposing 
an educational technology dilemma that required them to 
think critically about the topic, as they will inevitably be 
faced with making decisions about technology in their own 
classrooms in the near future. The goal in this research was 
two-fold: (i) to investigate the potential impact of online 
instruction using increased detailed instructions, instructor 
feedback and probing (treatment group) versus skeletal 
instructions and less than useful feedback and no probing 
from the instructor (control group); and (ii) to identify the 
similarities/differences within the treatment and control 


Control Groups Treatment Groups 

N=23 N=23 


Group Cl 

3 Female 

1 Male 

Group T1 

3 Female 

1 Male 

Group C2 

3 Female 

1 Male 

Group T2 

4 Female 

Group C3 

4 Female 

Group T3 

3 Female 

1 Male 

Group C4 

2 Female 

1 Male 

Group T4 

3 Female 

1 Male 

Group C5 

4 Female 

Group T5 

3 Female 

1 Male 

Group C6 

3 Female 

1 Male 

Group T6 

2 Female 

1 Male 


Table 1. Group Distributions by Size and Gender 


groups. If the authors prediction that the treatment group 
would excel was correct, they wanted to delve deeper into 
the different reactions between the treatment and control 
groups, In order to accomplish this goal the instruction and 
instructors involvement needed to be identifiably different 
for each group but the dilemma was the same for the 
treatment and control groups. The dilemma used in this 
study was chosen from a collection of controversial issues in 
education; should teachers adopt new technologies in K- 
12 schools (Abbeduto, 2000)? The online discussion took 
place over a two week period of asynchronous online 
discussions regarding the dilemma about educational 
technology in the schools. The guidelines were different for 
each week (see Appendix A for instructions for week 1; and 
see Appendix B for instructions for week 2). 

The dilemma and instructions for the control and treatment 
groups for week 1 were posted on the discussion board 
after a face to face meeting. The directions for week 1 were 
to discuss the dilemma asynchronously online. The 
treatment group received instructions to read the dilemma 
and post at least 3 times during the week following these 
specific guidelines: 

• Provide an initial posting (taking one side); 

• Then a challenge (debating a group member position); 
and 

• A response to a challenge (responding to the debate). 

The control group was given the same dilemma to read 
and simply asked to post 3 times with no other explanations 
(see Appendix A for instructions for treatment and control 
groups). 

At the end of the first week the leaders in the control and 
treatment groups were asked to provide their group with a 
brief summary of the discussion trend that developed over 
the course of the first week. During the second week of the 
asynchronous dilemma discussion the control and 
treatment groups were asked to respond to two items: (i) 
what is your reaction to the discussion topic as it evolved in 
your group; and (ii) what is your final resolution to the 
dilemma? 

In terms of the instructor's feedback to the groups, this was 
thoughtfully executed between the control and treatment 
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groups. During the first week of discussion, the treatment 
groups received brief but regular interaction from the 
instructor. The feedback from the instructor was supportive 
and directed toward individual responses, often it 
highlighted critical points that an individual made and 
aimed at posing more probing questions that required the 
groups to reflect on their knowledge and beliefs about 
educational technology. Below are four samples from the 
instructor's posting to different treatment groups, because 
there were six separate treatment groups the transcripts 
convey a variety of different comments but the nature of 
the instructor's responses remained consistently guiding, 
supportive, and probing. 

Sample 1 

I Encourage you to think and generate as many reasons as 
you can to support your claim. For example, if you are on 
the pro side of the issue try to go beyond what is presented 
in the dilemma by bringing in ideas from other classroom 
readings, your own experiences, and/or established facts. 
This would also be the case if you adopt the con side of the 
argument. When you do your second posting this week, 
which is the challenge to a group member's initial post, I 
encourage you to do the same. For instance, when you 
adopt the con side as a response to a pro side posting, 
make the case for why technology is so beneficial all the 
time by bringing in evidence from readings, experiences, 
facts etc. Even though you may not agree with the con side 
imagine that you do and think broadly and deeply for 
reasons and evidence that could be valid arguments for 
the position. Then in your third posting you can restate your 
belief again. They may be the same or you may see things 
somewhat differently after being engaged in this 
disagreement process. 

Sample 2 

These are very important concerns. It looks like you are not 
against new technologies but advocate for wise ways to 
use them. Any thoughts on what teachers can do to avoid 
technology dependency and still benefit from its use? 

Sample 3 

I am glad you took the con side. It is important to analyze 
both sides of the issue. You discuss most of the cons from 
your experience with online classes as a student. Flowever, 


if you put yourself in the role of the teacher, can you talk 
about any technological cons? For example, distance 
education entails more prep work on the instructors as they 
have to design the courses. 

Sample 4 

The Internet provides great resources but not everyone has 
access to it. Some parents may be able to buy a book for 
their children but not a computer. Even though they may 
have a PC, they might not have internet access. This is 
speaking in terms of families; but what about schools? 
There are many 'at risk' schools that do not have computers. 
Then we need to look at the schools who might have the 
technologies but do not use them; still kids do not get 
computer exposure either way. Do you think there is 
anything that can be done about availability of computers 
in schools? Flow do veteran teachers who have computers 
in their classrooms but lack knowledge and understanding 
to capitalize on the student learning? Can anyone find 
statistics about how many families or schools have or do 
not have access to computers or the Internet? 

During the first week of the discussion the control group did 
not receive any consistent feedback, it was generally once 
for each of the six control groups. The feedback did not 
address individual participants; it usually reiterated what 
was presented in a vague and generic manner. More 
importantly in the end it might be more important to notice 
what was missing in the feedback; such as, amount of 
instructor involvement, consistent and regular instructor 
interaction, void of positive comments, lack of specific 
examples or extrapolation of the student's thinking, and the 
absence of any type of probing questions. For example, 
this is an excerpt from instructor's feedback to a control 
group. 

Some good points being mentioned. Technology is 
already present, we just need to learn how to use it. 

During the second week of the discussion, again the 
control and treatment groups received different directions 
(see Appendix B for instructions). There was one other 
difference in the directions for week 2; the directions were 
not posted on the discussion board as with week 1, but 
rather appeared as informal discussion from the instructor 
in each group's small group discussion area. The feedback 
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for week 2 followed the same patterns for the treatment 
and control groups as they did for week 1. Here are 
abbreviated transcripts from feedback to the treatment 
and control groups for week 2. 

Treatment Sample 1 

This is an excellent point that is being made by a couple of 
the group members. It is true that today's students are more 
technologically advanced and educators have to try to 
keep up with that. Does this have to be a negative 
consequence? Can you talk more about benefits of 
students being more advanced than teachers? What 
might be some activities that could support technology if 
the students are more advanced? 

Treatment Sample 2 

This group is really exposing some great points! I am Glad 
that you are able to look in-depth at both sides of the issues 
that are being raised in your discussion. It seems that you 
are moving to the idea that technology in isolation is not so 
much a problem but how it is being utilized is what really 
makes the difference. What do you all think are some key 
elements to executing the use of technology? Do you see 
any differences in how this should be done for different age 
groups or to meet the individual differences of students with 
special needs or who come from culturally different 
backgrounds? 

Control Sample 1 

It helps move the discussion along if everyone participates 
in the discussion. I thinkyou all have some knowledge of this 
based on your personal experiences. 

Control Sample 2 

It could be that there are students who are not interested In 
technology and would rather not use a computer to 
Accomplish classroom tasks. 

Initial preliminary analysis was conducted using t tests to 
compare the two groups and support our qualitative 
investigation (quantitative results were reported in detail by 
Spatariu et al. 2007). In summary, argumentation was 
measured and results indicated subjects performed better 
both weeks in the treatment group (first week M= 3.088, 
SD= .361; second week M= 2.934, SD= .816; N=23)than 
in the control group (first week M= 2.845, SD= .464; second 


week M= 2.193; SD= .734; N=21). Interactivity was also 
measured and results indicated subjects performed better 
both weeks in the experimental group (first week M= 3.036, 
SD= .557; second week M= 2.978, SD= 1.060; N=23) than 
in the control group (first week M= 1.872, SD= .324; second 
weekM= 1.838, SD= .941; N = 21). However, the argument 
qualify was not statistically significant the first week even 
though the mean was higher in the treatment group (t (42| = 
1.942, p = .059). This prompted a more in-depth 
qualitative analysis of the content and qualify of the 
discussions, 

The qualitative analysis was done on 272 discussion 
transcripts generated by both groups over the two week 
duration of the study. Transcripts were coded based on 
group (treatment/control), small group (1, 2, 3 etc.), and 
participant; and uploaded into Atlas-Ti, a qualitative 
software program used for assisting in the coding, 
organization, analysis, and interpretation of the data. The 
researchers took a three step systematic design approach, 
open, selective, and axial coding to analyze the online 
postings (Creswell, 2008). Data was coded initially and then 
subsequently by two researchers for inter-rater reliability 
(.90). 

Results and Discussion 

In this section the researchers will report their findings for the 
treatment group in depth and provide the final analysis of 
the control group. The analysis procedure was conducted 
with three levels of content analysis; (i) conventional, (ii) 
directed, and (iii) summative (Hsieh & Shannon, 2005), The 
findings are rich in content, particularly for the treatment 
groups which demonstrated the most consistency and 
continuity within and between the six groups. Their belief is, 
this is the case due to the explicit guidelines and feedback 
that the treatment groups received compared to the 
control groups. All levels of analysis were initially conducted 
by the researchers and subsequently underwent two 
separate rounds of interrater reliability checking 
completed by graduate assistants working in the field of 
online discussions and argumentation but were not 
involved with the data collection or the course. 

Level 1 analysis used the conventional approach to 
organize the large amount of data. The conventional 
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content analysis uses the text from the data to identify 
codes for a particular way of thinking or position which was 
later able to be placed in larger categories. Level one 
conventional content analysis was completed on all 272 
posted responses which resulted in 84 codes emerge from 
the (see Appendix C). In appendix C the numbered items 
reflect the initial level of coding and represent that all items 
were identified in the treatment and control groups. These 
items were further reduced into categories (see Appendix 
C; lettered and bolded items). Appendix C is presented to 
demonstrate the rigor of the qualitative analysis and to 
allow other researchers to use a similar coding scheme in 
future research; space does not permit extrapolation of the 
coding scheme. They found that the responses had a high 
degree of evaluative characteristics (especially within the 
treatment groups), that is, the responses were not 
dichotomous; such as, simply a pro or con for educational 
technology. Instead, the breadth of participant responses 
went beyond; to include a pro and con label; in addition 
they included the following: don't know, don't care, could 
be both, and it all depends. Later the 6 initial coding labels 
for the category used as the example above were 
reduced to the following 4 labels: (i) pro, (ii) con, (iii) don't 
know and don't care; and (iv) could be both and it 
depends.This approach to content analysis is a mixture of 
inductive and deductive perspectives, below are a 
sample of the coding from the original transcripts. 

TEBSP: Treatment Group Evaluation Based on Student 
Perspective 

I am definitely for using technology to help teach students 
in the classroom. The fact that students can be figuring 
things out for themselves can be much more beneficial 
than someone simply showing them how to do it. 

TEPE: Treatment Group Evaluation Based on Personal 
Experience 

Even though I took the con side, at home I am addicted to 
the internet and other computer software! I do learn things 
that I do not learn in class but sometimes it keeps me from 
doing what I need to do and I am not focused on what the 
teacher wants me to focus on. Students need to stay 
focused and I see computers and other technology as a 
big distraction for today's young students. 


TB: Treatment Group Both (it depends] 

Let me clearly state that I do not believe that technology 
should be the only method of instruction in order to yield 
desirable results. It all depends on how we choose to 
incorporate it that could potentially make the difference. 

Level 2 was primarily deductive directed content analysis 
based on preliminary quantitative data analysis; and 
categories found in the research (see Appendix C). In a 
directed content analysis the researcher begins with a 
theory or relevant research as a driving force for the initial 
coding of large amounts of data (Hsieh & Shannon, 2005). 
In appendix C we identified 5 categories that appear as 
boldface: (i) position on dilemma; (ii) support of position on 
dilemma; (iii) perspective of dilemma evaluation; (iv) 
nature of dilemma evaluation; and (v) nature of response 
to others. For example, using the quotes used in level 1; 
TEBSP and TEPE were place under the category of 
Perspective of Dilemma Evaluation; and TB was 
categorized under Position on Dilemma. Once all codes 
were evaluated and they were assigned to a category, all 
codes for each category were counted and ranked 
highest to lowest within each category for each group. 
Then the rankings were compared between all of the 
treatment groups and all of the control groups. The findings 
of the study show that there were similar rank orders for the 
categories of most of the treatment groups but not the 
control groups. Out of the six treatment groups the 
researchers found 5 groups that had very similar trends in 
their rankorders in three of the categories: 

• Perspective of dilemma evaluation (Figure 1); 

• Nature of dilemma evaluation (Figure 2); 

• Support of position on dilemma (Figure 3). 

Perspective of dilemma evaluation comprised the 
arguments that had to do with how the student presented 
support for their position on educational technology. This 
category encompassed arguments that were set in the 
context of personal experience (54%), background 
knowledge (28%), based on a teacher's perspective (9%), 
or from a student's perspective (18%) (Figure 1). Treatment 
group 6 was the only treatment group that had a different 
ranking in this category. It was primarily the females in this 
group that shifted the results; they used more information 
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■ Personal 
Experience 

Background 

Knowledge 

■ Teacher 
Perspective 

■ Student 
Perspective 


Figure 1. Treatment Perspective of Dilemma Evaluation 


from the text and background knowledge as opposed to 
personal experience. This could be because admittedly 
they disclosed they were not computer savvy; they found it 
to be cumbersome and time consuming. Treatment group 
6 also had the bulk of the student perspective responses. 
Interestingly, the responses that were derived from a 
teacher perspective were the least represented. We think 
this may be because of the undergraduate status of the 
students and the lack of teaching experience. It is standard 
for pre-service teachers to view themselves in the role of the 
teacher; and conversely identify with a student role which 
they are currently experiencing. The control groups varied 
drastically in this category, there was little continuity 
between the groups and typically each of the control 
groups took on the perspective of their leader. That is, if the 
leader used a personal experience, the others followed in 
suite. Two of the control groups were identified as personal 
experience, three groups supported their positions with 
background knowledge or more objective text and 
research materials, and one group represented the 
student viewpoint in their discussions. The large amount of 
text-based background knowledge responses indicated a 
lack of individual thinking and a lack of critical perspective¬ 
taking that appeared to be present in the treatment 
groups. The control groups had less interaction and the 
discussion was more linear. 

Nature of dilemma evaluation comprised excerpts of the 
discussions that could be labeled as cognitive (39%), 
solution-oriented (or metacognitive) (31%), emotional 
(26%), and social (14%) (Figure 2). The researchers saw the 
exact same trend in all six of the treatment groups. 
However, the trend in this category was more consistent 
among the control groups; it was not remotely consistent 


with the treatment group. The nature of the dilemma for the 
control groups were based primarily on and fueled by 
social and emotional responses. There were no varying 
levels of cognitive perspective as seen in the treatment 
groups, Again, this speaks to the linear course of the control 
groups' discussions and this category demonstrated the 
lack of evaluation between the objective and subjective 
perspectives that lead to proper evaluation of a dilemma. 
The control groups discussion were more opinion and not 
well developed for adequate solutions; this was promptly 
visible in the week 2 discussion of the control groups as seen 
in the short responses and frequency of responses. The 
treatment groups all had the same rank order and the 
nature of the dilemma was more evenly distributed among 
the treatment groups; typically a response would cover two 
or more of the areas in this category. For example, a 
student would begin with lower level thinking and develop a 
train of thinking that would consist of higher-order thinking 
(application, synthesis, evaluation). The responses of the 
treatment groups appeared to be more organized 
(another indication of critical thinking); integrated amongst 
group members (they were interacting and responding to 
each other), and the trajectories of each groups discussion 
was solution-oriented. This level of thinking set the groups up 
nicely for the second week of discussions, They had more 
exhaustive debates on both sides of the dilemma because 
they had to look at alternatives that they may or may not 
have agreed with because they were directed to respond 
and argue the opposite point of their group-members, This 
clearly had an impact on the sophistication of the 
treatment group's discussions, As indicated earlier, there 
were clear complexities in the nature of evaluation in the 
treatment groups as was seen in the integration of domains 



■ Cognitive 
Solution Oriented 

■ Emotional 

■ Social 


Figure 2. Nature of Dilemma Evaluation 
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within single responses. That is, a single response entailed 
not only cognitive views but integrated views that included 
emotional and social dynamics; this was more the norm 
among the treatment group's responses. 

Support of position on dilemma was comprised of the 
responses according where the individuals stood on the 
dilemma regarding educational technology in the 
schools. This category looked at two variables; whether 
they were for or against educational technology and 
whether or not they provided explicit examples for their 
position. The distribution was as follows: examples for 
support (71%), no examples for support (4%), examples 
against (9%), and no examples against (6%) (Figure 3). Five 
of the treatment groups presented the same rank order for 
this category. The outlier in this category was treatment 
Group 3; which consisted of 4 females who were more 
against educational technology. The interesting part of 
treatment Group 3 being the outlier in this category was 
that the bulk of the social response in the Nature of the 
Dilemma Evaluation category were derived from Group 3 
participants as the primary reason they were against 
educational technology. Even more interesting was that 
when the other treatment groups provided examples for 
supporting educational technology one of the most 
prominent reasons for supporting it was because they 
believe in a constructivist approach to learning or they 
favored a Vygotskian approach which included peer or 
collaborative learning. Now two things are important to 
mention here; when these were the emphasis of their 
argument it was coded as cognitive because of the 
application of learning theory; but one could also have 
interpreted that they were getting at a social perspective 

■ Examples for 
Support 

No Examples for 
Support 

■ ExamplesAgainst 

■ No Examples 
Against 

Figure 3. Support of Position on Dilemma 


because that underpins Vygotskian and constructivist 
approaches. They contemplated this issue and decided 
that their intention was not so explicitly social but more 
entailed more critical thinking and reflection but the 
researchers mention it here because it could be an 
argument for a social perspective. Therefore, their first 
finding for this category was that the treatment group had 
a significantly broader and more in-depth view of the 
dilemma due to the overwhelming support of their 
responses by providing examples for their position. The 
second finding was the treatment group overwhelmingly 
supported educational technology in schools. On the other 
hand, the control groups, who were more mixed on the 
dilemma with regard to being for or against educational 
technology. In addition, the control groups demonstrated 
more limitations to their discussions by having far less 
support for their positions. The control groups floundered by 
comparison to the treatment groups in this category by far 
the most. They did not have enough depth or breadth in 
their discussions to justify support for educational 
technology and it was primarily driven by emotional 
responses to the dilemma. 

Level 3 analysis procedures were conducted to show the 
process of the discussions between the treatment and 
control groups. The frequency distributions do not enable, 
however, the examination of the evolution of the process 
(e.g., changes in the knowledge production over time, 
differences in discourse patterns, or the role of individual 
ideas in broader context). The researchers used more 
descriptive methods to evaluate the differences in the 
nature and style of the knowledge production in the 
differing conditions. For this purpose, the unit of analysis was 
extended to cover the entire material for each group to 
enable a process perspective on the discourse. Small 
group data was merged and recoded using the more 
narrow categories. Consequently, the process analysis 
presents a synthesis of what were considered distinguishing 
features of the process in between group differences. The 
following features were judged as representing plausible 
operations, background knowledge, personal experience, 
explanation process, developing ideas in dialogue, and 
self-reflection. 
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It was clear by the findings in level 2 analysis that the 
treatment groups out preformed the control groups. Also, 
the researchers identified patterns among the treatment 
groups but they did not find any coherent patterns among 
the control groups, This raised the curious question about 
the discourse processes that occurred between the control 
and treatment groups. They conducted a summative 
content analysis, which is inductive, guided by emerging 
categories in the data. Here they reduced the data further 
by collapsing the treatment groups and the control groups 
as a means of identifying a broader perspective about the 
course or process on the discussions as they related to the 
larger categories. In other words, they wanted to know how 
the larger categories were represented in the treatment 
group versus the control group. Summative content 
analysis involves counting and comparing groups within 
and between by using keywords or content, followed by 
interpreting the underlying context. Using this method of 
analysis patterns emerged in terms of how the categories 
were addressed and evolved throughout the discussions. 
This provided an overview about the nature of students' 
knowledge, critical thinking, argumentation, and counter¬ 
argumentation. 

The important finding identified in the level 3 analysis was 
the clearly different patterns that emerged between the 
treatment and control groups regarding how the 
participant's thinking about the dilemma evolved 
throughout the discussion. Evaluation of the treatment 
groups indicated a more complex and sophisticated 
developmental pattern that took seven steps to achieve a 
viable solution to the dilemma of educational technology 
in the schools (Figure 4). The treatment group appeared to 
be manipulating the information that was presented in the 
discussion to achieve a solution; and perhaps the 
participants had a notion or understood that they may be 
heading for a change in their perspective based on the 
information that was produced during the discussion. 
Although, it is thought through our analysis that even though 
they may have anticipated some change in their 
viewpoint, there was an underlying sense of ambivalence 
and some amount of struggle during the process. This 
apparent struggle was noticeable at this level of analysis 
because all groups revisited two of the larger categories 
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Figure 4. Treatment Group Discourse Process 

identified in level two analysis; perspective of dilemma 
evaluation and support of position on dilemma. 

Figure 4 helps to explain the treatment groups discourse 
process as it unfolded. The groups began by, first, providing 
a perspective of the dilemma, second, they moved to 
providing some indication about how they viewed the 
nature of the dilemma, third, each member of the group 
provided support for their perspective, and then fourth, 
each member progressed to responding to other's. Rather 
than continuing to proceed with the goal of the assignment 
(provide solutions for the dilemma), the treatment groups 
consistently across all 6 groups retreated back to rehashing 
their position on the dilemma; and then again revisited 
support for their position prior to indicating a change. From 
Figure 4, step 1 and 5 are the same but we chose to name 
step 5 slightly different to distinguish the recycling back to 
this category. Although the topic was the same, there were 
some definite differences the second time around. For 
example, this is where many of the responses for the 
treatment group's emotional nature of the dilemma were 
identified; and the length of the responses were more brief 
but straight to the point at hand; and this is also a section of 
the discussion in which responses were posted without 
support. 

The last of the differences (posting without support) is 
particularly interesting because it is the second category to 
be revisited by the treatment group. From Figure 4, steps 3 
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and 6 are the same; they chose to use the same label 
because there was no apparent difference in the 
discussions. What did occur here was that initially when the 
participants supported their perspective is was generated 
by the individuals, but when it was revisited in step 6 it was 
because other group members pointed out that some 
members were posting perspective without support and 
responded accordingly to request that support be 
provided. Where individuals did not generate the support, 
their peers activated a request for support and so the 
category was revisited. These 6 steps prompted some 
changes to individual's responses to the educational 
technology dilemma, some changes were drastic and 
others were mild to moderate.. 

The control group's discussion patterns were more 
abbreviated in the process. Like the treatment group, the 
control group demonstrated overlapping characteristics in 
the content of the discussions and patterns in the 
discussion; however they could only uncover three stages 
within the control group's discussions (Figure 5). The first 
stage in the discussion was shared among two of the 
categories identified in level 2 of the content analysis; 
perspective on the dilemma and support for perspective; 
the second stage in the development of the discussion was 
the nature of the dilemma; and third nature of the response 
to others. 


Addressing the first stage which is shared by two categories 
is important because it occurred often in the control groups 
but not at all in the treatment groups. Frequently, 
participants would not state their own perspective but 
rather they adopted the perspective of the previously 





Figure 5. Control Group Discourse Process 


posted response and proceeded to discuss support for 
another participant's perspective. It was not uncommon to 
see initial postings that stated a perspective without any 
support. This lack of support may have prompted the other 
students to feel the need to provide support for another 
participant's perspective due to the absence in the original 
posts. In general, the control groups had little organization 
that could be identified in their responses. There seemed to 
be disjointedness in the control groups, in that there was not 
movement or advancement in the discussions. This made 
the findings of the third stage (nature of response to others) 
not surprising on one hand but somewhat perplexing on the 
other hand. The nature of response to others for the control 
group was completely dichotomous to the treatment 
group. In the control groups there were no challenges to 
others comments, no matter how poor in thought quality 
they may have been; there were detachment among 
posted responses that clearly did not even align with any 
post previously made in the groups. It was as if the individual 
thought of the post in advance and posted it regardless of 
what anyone else had to say. There were no attempts do 
any problem solving or identify possible solutions, in fact, 
the underlying feeling of apparent change that was 
prominent in the treatment group was not even remotely 
scratching the surface in the control groups. The emphasis 
was more on opinions and subjectivity than on objective 
knowledge; this is interesting because they seemed to use 
the text as a point of reference but there was no ownership 
of the content. There was an absence of social and 
emotional domains aligned with their low level cognitive 
responses. 

The researchers believe the profound differences that were 
identified in the control groups, as compared to the 
treatment groups, on an individual group basis and 
collectively were strongly indicative of the lack of specific 
guidelines provided to the control groups. Remember that 
the control group was not given directions to think or reflect 
in a metacognitive manner. In addition, the control group 
received only minimal vague feedback from the instructor. 
This is a good indication that instruction during the use of 
educational technology platforms may require both 
detailed instruction; and constant and consistent 
feedback from the instructor. 
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Discussion and Conclusion 

The selected dilemma pertained to the level of the 
cognitive (topic appropriate for discussion in an 
undergraduate educational technology course), social 
(an issue that directly concerns current schooling system), 
and domain level of the subjects (the issue concerns them 
directly as future teachers faced with the rapid 
advancement of learning technologies) thus meeting 
basic argumentation requirements (Coirier et al., 1999; 
Golder & Pouit, 1999; Stein & Miller, 1991), This analysis 
indicated that the treatment group outperformed the 
control group in every category consistently within the small 
group analysis and the between group analysis; and 
provided richer text in order to specifically identify the 
following main themes: Position on Dilemma; Support of 
Position on Dilemma; Perspective of Dilemma Evaluation; 
Support on Perspective of Dilemma Evaluation; Nature of 
Dilemma Evaluation; Nature of Response to Others; Nature 
of Position Change. 

The primary research questions involved with the present 
study were: (i) What are participants' reactions to the 
educational technology dilemma within and between 
groups; and (ii) are the reactions within and between 
groups different? If so, how are they different? The above 
analysis of the discussions shows what the particular 
reactions were and how they differ at both group levels, 

In summary, first, the process between the treatment group 
and control group, demonstrated more sophisticated 
critical thinking in the treatment group. The most distinctive 
result is the emphasis between the perspective-taking 
(opinion-based) and deciding on a position (evidence- 
based); and the nature of the support. The treatment 
group progressed through a series of discourse and 
evaluation (7 steps); whereas, the control group functioned 
much less critically (3 steps) and rarely settled on a position 
but rather provided more opinion than evidence based 
perspectives. 

Second, the treatment group reflected more perspectives 
initially and interacted with their peers prior to settling on a 
position; and because of this some students indicated a 
change of perspective. The control group never indicated 
a change in their perspective. 


Third, support in the treatment group, reflected 3 main 
areas in a clear hierarchical manner: (i) personal 
experience; (ii) use of technology examples; and (iii) 
background knowledge; and rarely based support on 
teacher or peers. The control group lumped their 
perspective and support in a single effort; and generally 
based on personal experience and peers (often no support 
was ever provided), Additionally, some gender differences 
were observed but more research would be necessary to 
make valid inferences since there were very few males 
present in all groups. 

This study realizes the complexity and rigor of a qualitative 
analysis in this area. This study summarizes categories that 
may promote or inhibit higher order thinking during on-line 
discourse and demonstrates that discussion leader 
prompts can generate better arguments and increase the 
interactivity within small group online discussions, When 
instruction is structured and goal oriented students thinking 
is less impacted by the instructor's comments. They also 
recommend training students in being the discussion 
leaders within the groups. That way they are more 
empowered and overtly trained in critical thinking 
engagement and strategies. From a research perspective 
there may also be a difference in between student lead 
discussion and instructor lead discussion as students 
interact differently with their peers. 

In terms of future research, they recommend a more in- 
depth look at the, nature of the responses category in order 
to better evaluate the cognitive, social, and emotion 
facets of the two groups (e.g., have the students write a 
reflective paper or take a survey specifically geared to 
these areas). This may provide a richer understanding of 
the differences that underlie critical thinking skills, 
argumentation, and decision-making/reasoning abilities. 

With the rapid advancement of technologies, another 
important future recommendation for research is to 
automate prompts by programming them in the 
discussion board software, For example, a more intelligent 
system can be built, based on key critical thinking words, 
which give students prompts such as metacognitive 
questions while they write their posts. 
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AppendixA 

Instruction given by the discussion group leader to the 
treatment group: 

• Read the technology dilemma carefully. You have to 
do at least 3 posts for this week's discussion (initial, 
challenge, and response to challenge). 

• Make the first posting. In this first posting you will adopt 
the side of the dilemma you mostly agree with (either the 
pro or the con presented). State your claim clearly and 
support it with evidence and/or reasoning. 

• Read other group members postings. Respond to at 
least one of them. The response has to be a challenge to 
the posting you are responding to. The challenge means 
adopting and supporting with evidence the opposite side 
of what is presented in the posting you are responding to. 

• Make a third posting. This will be a response to one of 
the challenges addressed to you. 

Instructions given by the discussion group leader to the 
control group: 
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• Read the technology dilemma carefully. 

• You have to do at least 3 posts for this week's discussion. 

• Make an initial posting as an answer to the dilemma, 

• Respond to at least 2 other people postings, 

Appendix B 

Instructions given to the treatment groups for week 2 
dilemma discussion: 

• Here is what we have to do this week. After discussing 
the dilemma of whether new technologies should be 
adopted by the schools or not; a certain group direction 
has emerged. Read your group leaders summary carefully 
and respond to the following to questions. 

1, What is your reaction to the discussion topic as it evolved 
within the group? 

2. What is your final resolution to the dilemma? 

• Respond to each of these two questions in separate 
postings. First, post your response to question 1, then, wait a 


day or two before you post your response to question 2. By 
waiting to post your response to question 2, you will have 
the opportunity to read other group members' reaction to 
the group direction. Take the time to reflect on your initial 
thoughts and the reaction of your group-mates; think 
through your response thoroughly using your abilities to be 
critical thinkers. 

Instructions given to the control groups for week 2 
dilemma discussion: 

• Here is what we have to do this week. After discussing 
the dilemma of whether new technologies should be 
adopted by the schools or not; a certain group direction 
has emerged. Read your group leaders summary carefully 
and respond to the following to questions. 

1. What is your reaction to the discussion topic as it evolved 
within the group? 

2. What is your final resolution to the dilemma? 


Appendix C 



CODES 

MEANING 


CODES 

MEANING 

A 

Control Position on Dilemma 


AA 

Tx Position on Dilemma 


1 

CPRO 

Control Pro (for computer technology) 

43 

TPRO 

Treatment Pro (for computer technology) 

2 

CCON 

Control Con (against computer tech.) 

44 

TCON 

Treatment Con (against computer tech.) 

3 

CN 

Control Neutral (don't care, don’t know) 

45 

TN 

Treatment Neutral (don't care, don’t know) 

4 

CB 

Control Both (could be both, it depends) 

46 

TB 

Treatment Both (could be both, it depends) 

B 

Control Support of 

Position on Dilemma 


BB 

Tx Support of Position 
on Dilemma 


5 

CSUPPRO 

Supports Argument for Tech. 

47 

TSUPPRO 

Supports Argument for Tech. 

6 

CNSUPPRO 

No Support of Argument Tech. 

48 

TNSUPPRO 

No Support of Argument Tech. 

7 

CEXSUPPRO 

Uses Examples for Support Tech. 

49 

TEXSUPPRO 

Uses Examples for Support Tech. 

8 

CNEXSUPPRO 

No Use of Examples for Support Tech. 

50 

TNEXSUPPRO 

No Use of Examples for Support Tech. 

9 

CSUPCON 

Supports Argument for Against Tech. 

51 

TSUPCON 

Supports Argument for Against Tech. 

10 

CNSUPCON 

No Support of Argument Against Tech. 

52 

TNSUPCON 

No Support of Argument Against Tech. 

11 

CEXSUPCON 

Uses Examples for Support Against Tech. 

53 

TEXSUPCON 

Uses Examples for Support Against Tech. 

12 

CNEXSUPCON 

No Use of Examples for Support 

Against Tech. 

54 

TNEXSUPCON 

No Use of Examples for Support Against Tech. 

C 

Control Perspective of 
Dilemma Evaluation 


CC 

Tx Perspective of 

Dilemma Evaluation 


13 

CEPE 

Control Evaluation Based on 

Personal Experience 

55 

TEPE 

Tx Evaluation Based on Personal 

Experience 

14 

CEBK 

Control Evaluation Based on 

Background Knowledge 

56 

TEBK 

Tx Evaluation Based on Background 
Knowledge 

15 

CEBTP 

Control Evaluation Based on 

Teacher Perspective 

57 

TEBTP 

Tx Evaluation Based on Teacher 

Perspective 

16 

CEBSP 

Control Evaluation Based on Student 
Perspective 

58 

TEBSP 

Tx Evaluation Based on Student Perspective 
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D 

Control Nature of Dilemma Evaluation 

DD 

Tx Nature of Dilemma Evaluation 

17 

CEC 

Control Evaluation Cognitive 

59 

TEC 

Tx Evaluation Cognitive 

18 

CEE 

Control Evaluation Emotional 

60 

TEE 

Tx Evaluation Emotional 

19 

CES 

Control Evaluation Social 

61 

TES 

Tx Social 

20 

CEPRB 

Control Evaluation Problem 

62 

TEPRB 

Tx Evaluation Problem 

21 

CESOL 

Control Evaluation Solution 

63 

TESOL 

Tx Evaluation Solution 

22 

CER 

Control Evaluation Reasoning 

64 

TER 

Tx Evaluation Reasoning 

23 

CECT 

Control Evaluation Critical Thinking 

65 

TECT 

Tx Evaluation Critical Thinking 

24 

CEDM 

Control Evaluation Decision Making 

66 

TEDM 

Tx Evaluation Decision Making 

25 

CECMPR 

Control Evaluation Compromise 

67 

TECMPR 

Tx Evaluation Compromise 

E 

Control Nature of Response to Others 

EE 

Tx Nature of Response to Others 

26 

CRCH 

Control Response Challenging 

68 

TRCH 

Tx Response Challenging 

27 

CRCHM 

Control Response Challenge Multiple Points 69 

TRCHM 

Tx Response Challenge Multiple Points 

28 

CRU 

Control Response Understanding 

70 

TRU 

Tx Response Understanding 

29 

CRA 

Control Response Aggressive 

71 

TRA 

Tx Response Aggressive 

30 

CRD 

Control Response Detached 

72 

TRD 

Tx Response Detached 

31 

CRC 

Control Response Cognitive 

73 

TRC 

Tx Response Cognitive 

32 

CRE 

Control Response Emotional 

74 

TRE 

Tx Response Emotional 

33 

CRS 

Control Response Social 

75 

TRS 

Tx Response Social 

34 

CRPRB 

Control Response Problem 

76 

TPRB 

Tx Response Problem 

35 

CRSOL 

Control Response Solution 

77 

TSOL 

Tx Response Solution 

36 

CRR 

Control Response Reasoning 

78 

TR 

Tx Response Reasoning 

37 

CRCT 

Control Response Critical Thinking 

79 

TCT 

Tx Response Critical Thinking 

38 

CRDM 

Control Response Decision Making 

80 

TDM 

Tx Response Decision Making 

39 

CRCMPR 

Control Response Compromise 

81 

TCMPR 

Tx Response Compromise 

F 

Control Nature of Position Change 

FF 

Tx Nature of Position Change 


40 

CCGG 

Control Change 

82 

TCGG 

Tx Change 

41 

CCGL 

Control Change Based on Logic 

83 

TCGL 

Tx Change Based on Logic 

42 

CCGNL 

Control Change Based on NO Logic 

84 

TCGNL 

Tx Change Based on NO Logic 
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